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Executive Summary 


This report conveys the outcomes of a value-added analysis of Grade Level of Achievement 
(GLA) data at the provincial level. GLA data reported to Alberta Education is a teacher’s 
judgment of a students’ academic progress. GLA is based on the learner outcomes in a subject area 
after a course for a specific grade level has been completed and reflects the results from the full range 
of classroom assessments. Given the comprehensiveness of classroom-based assessment, analysis 
of GLA data can provide unique insights into factors that influence student achievement. GLA 
data provides teachers, principals, central office staff and Alberta Education with an additional 
tool to help inform and engage students and parents in the learning process, and to analyze and 
evaluate the achievement of different populations of students to ensure that their learning needs 
are better understood and met. When GLA is reported to parents and receiving teachers this 
information benefits students by supporting accurate and timely communication of achievement 
information as they move from one grade to the next or between schools. 


Alberta Education will use GLA data for program planning. The data is not used as part of the 
Accountability Pillar. The key purpose of reporting GLA is identifying students who are under- 
achieving, asking why and providing solutions both individually and systemically. 


Background 


GLA pilots to demonstrate and build GLA reporting capacity were run in 2003-04 and again 
with a larger set of schools in 2005-06. Detailed analysis of the outcomes of these pilots 
supported the reliability and validity of the initial GLA data collection initiatives (Alberta 
Education, 2005, 2007) and provided support for continuing with the implementation of the GLA 
initiative. 


This report describes the processes and outcomes associated with the 2006-07 Grade Level of 
Achievement (GLA) pilot data collection, data management and data analysis. Results of this 
and previous studies of GLA pilots will inform options for adjusting strategies for provincial 
implementation of the GLA reporting initiative. The current report demonstrates support for the 
four purposes for reporting GLA as defined in the GLA Handbook (Alberta Education, 2006:4), 
specifically: 

e to provide richer information at the system level (both jurisdictional and provincial) to 
inform effective practices to determine the impact of specific programs on student 
learning (e.g., English as a Second Language, special education) and to determine 
processes to further refine these programs; 

e as a catalyst within the school’s professionali learning community to focus on individual 
student learning needs and interests; 

e to determine effective practices and strategies to foster higher levels of student 
achievement and confidence; and 

e to contribute to the data or evidence used to report student achievement to 
parents/guardians, fulfilling the school’s responsibility as outlined in the Guide to 
Education: ECS to Grade 12 in the section entitled Assessment as the Basis for 
Communicating Individual Student Achievement. 


Study Limitations 


Reporting of GLA to Alberta Education in 2006-07 was expected by one-third of the Grade 1-9 
schools in the province. This expectation was substantially exceeded with GLA data being 
submitted for the students in 60% of schools. The higher than expected number of students for 
whom GLA has been reported should still be treated with some caution, as there is still the 
potential for sampling bias which must be taken into consideration when reading this report. 


The data used in this report were extracted from the data warehouse at the end of September in 
order to support the timely production of this report. Some GLA data has since been updated in 
the warehouse, that is, reports viewed on the Extranet by jurisdiction personnel may have slight 
variances with the results shown in this report. 


Key Findings from the 2006-07 Analysis 
e The degree of jurisdiction participation in the 2006-07 pilot was much higher than expected. 
A participation rate of 33 % was expected; however, the data submitted represents 60% of 


schools. 


e The error rate for data submission was low with only 2.2% of the files submitted having 


errors. 


e Compared to the 2005-06 data, the 2006-07 data demonstrates similar results for the total 
cohort and for specific sub-groupings of GLA data. Patterns of data distribution observed in 
2005-06 with 82,000 students were also observed in the 2006-07 data. 

e The data analysis demonstrates many interesting relationships. For example: 


fe) 


The large difference between Math 9 Provincial Achievement Test data and GLA 
data continues to be evident. . 

A much higher percentage of students coded as gifted are assessed above grade 
level in Math than in English Language Arts or French Language Arts. 

When comparing foreign-born and Canadian-born students, both groups of 
students perform similarly in Mathematics but in English Language Arts, 
Canadian-born students are at an advantage. 

Although a serendipitous finding, there is a disproportionate ratio of male: female 
severely coded students. There are nearly four times as many males as females 
coded as severely disabled. 

When comparing GLA and PAT data, females outperform males in GLA to a 
statistically significant degree in both subjects in nearly all enrolled grades. The 
reverse is seen on the Math PAT results with males outperforming females to a 
statistically significant degree. In English Language Arts, females are 
outperforming males on the PATs. 

A greater proportion of high mobility students (i.e. those who have changed 
schools more frequently than other students) have a GLA below their enrolled 
grade level. 


e When examining mean GLA compared to enrolled grade, interesting patterns emerge: 


° 


Foreign-born and Canadian-born ESL students’ GLA distributions are much 
closer together this year than was seen in the 2005-06 data. 


o Foreign-born ESL students seem to have a GLA on par with their enrolled grade 
in Mathematics up until Grade 5. In English Language Arts the two measures 
diverge as early as Grade 2. 

e The age effect’ is apparent in English Language Arts in Grades | through 5. After Grade 5, 
the age effect tapers off and is no longer apparent. This relationship is more pronounced in 
the 2006-07 GLA data than in 2005-06, possibly due to the larger number of students in the 
2006-07 data. 

e There are moderate strength correlations between PATs and GLA (Grades 3, 6 and 9). This 
demonstrates a reasonable degree of concurrent validity of the GLA data. 

e Using GLA and PAT information, GLA can provide important information for students that 
would not otherwise be available for students in grades not tested by PATs. 


How will GLA Information be used? 


e The GLA data is intended for use by Alberta Education, school jurisdictions and schools 
to better inform parents of their child’s progress. 


e Examples of Alberta Education data usage are: 


o To assist in planning improvements to provincial initiatives or programs and for 
curriculum evaluation. 

o To conduct further research into understanding how factors such as student 
gender, age, mobility and socio-economic variables influence student 
achievement and how interventions strategies can mitigate such negative effects. 

- This research will be shared with jurisdiction staff to inform local strategies for 
improving student achievement. 


e School jurisdiction and school usage include: 


o GLA data will help to ensure students and parents know and appreciate the grade 
level of the program of studies that the student has achieved. This information 
can also assist grade or school transitions so that programs are planned based on 
the demonstrated achievement to date. 


o Using GLA data as a benchmark, jurisdictions may wish to compare the data in 
school and jurisdiction GLA reports over time or in relationship to provincial 
norms. This can support conversations in professional learning communities 
regarding promising practices that have been demonstrated to improve student 
achievement in specific settings. 


e In addition, GLA data collected by Alberta Education will be available upon request to 
schools as a backup if a student’s record is delayed in transit or lost. This service will 


' Age effect is defined as older students in a grade tending to have higher average test scores than the younger students in that 
same grade when measured by the z-score of average PAT results for each birth month group (Alberta Learning, 2001). 


ill 


help teachers ensure their instruction is geared to the student’s instructional level and will 
help the student’s transition to their new school. 


Next Steps in Implementation 


Current plans call for submission of GLA data in Language Arts and Mathematics by all Grade 
1-9 schools in the province at the end of the 2007-08 school year. Plans are on track for 
receiving this data with zero errors. Production of GLA reports will provide schools and 
jurisdictions with analysis of their GLA data in relationship to their students’ age, gender, 
mobility and a range of socio-economic variable that will assist staff in gaining additional insight 
into how these variables influence their students’ achievement. These reports will be available 
via the Extranet to the submitting schools and jurisdictions by mid-October 2008. The provincial 
report summarizing the 2007-08 GLA data will have a strong focus on how well various 
programs are serving Alberta students. 


Introduction 


This report describes the processes and outcomes for the 2006-07 Grade Level of Achievement 
(GLA) pilot data collection, data management and data analysis. The report is also intended to 
define the provincial context supporting the four purposes for reporting GLA as defined in the 
GLA Handbook (Alberta Education, 2006:4), specifically: 

e to provide richer information at the system level (both jurisdictional and ee, to 
inform effective practices to determine the impact of specific programs on student 
learning (e:g., English as a Second Language, special education) and to determine 
processes to further refine these programs; 

e asa catalyst within the school’s professional learning community to focus on individual 
student learning needs and interests; 

e to determine effective practices and strategies to foster higher levels of student 
achievement and confidence; and 

e to contribute to the data or evidence used. to report student achievement to 
parents/guardians, fulfilling the school’s responsibility as outlined in the Guide to 
Education: ECS to Grade 12 in the section entitled Assessment as the Basis for 
Communicating Individual Student Achievement. 


Limitations of the Data 
When analyzing the data, the following limitations were noted: 


e In 2006-07, a much bigger sample than expected was submitted that is somewhat 
representative of the full population of Grade 1-9 students, however, some degree of 
sampling bias should be assumed. 


e Generalizations should be considered in relationship to the estimated confidence intervals 
detailed on page 7. 


GLA 2007 Data Collection 


Given that this was still a pilot year, a remarkably low number of data transmission errors 
occurred during this year’s GLA data collection. Of a total of 228,150 GLA records submitted, 
there were 5,023 errors received (2.2%). The following table lists the number of errors by 
jurisdiction and the number of schools affected by these errors. 


# of Schools Affected 


The following list shows the types of errors that occurred as well as the number and percent of 
each type of error. 


[ErrorType sd Number | Percent | 


Grade English Language Arts Introduced column incorrectly filled in 2,377 | 47.3% 
for FLA students 


Duplicate record of learner (i.e. two entries for the same person 1,417 
GLA/IPP codes not being exclusive 1,175 23.4% 
Incorrect GLA or IPP code 48 


8 
Learner’s ASN (Alberta Student Number) not found T 6] 0.1% | 
5,023 | 100.0% 


The Edulink software and manual used to transmit GLA data to Alberta Education has been 
upgraded to identify solutions to errors that exist in the data files when they are transmitted to 
Alberta Education. The GLA-Edulink Manual will be revised to require errors be corrected 
before the deadline for GLA data submissions. This step is expected to reduce the error rate to 
0%. 


The individual student records having errors were not included in the 2006-07 data throughout 
this report nor in the school and jurisdiction reports available on the Extranet. School authorities 
were advised of the specific error reports so they could accommodate the error rate in their 
interpretation of the value added GLA reports provided back to the jurisdiction and schools that 
submitted GLA data. A total of 5,023 files were excluded from the data analysis due to errors. 


? In this particular jurisdiction the bulk of the errors occurred in three schools, the remainder of schools had only one 
or two errors per school. 


Adjustments for GLA 2008 Data Collection 


Quality control measures have been built into the Edulink reporting system. In 2007-08, 
jurisdictions submitting files that contain errors will be sent an automated email indicating the 
files containing errors. Files containing errors are to be amended and re-sent via Edulink by July 
7, 2008. 


Description of GLA Data 


A total of 923 schools from 71 authorities submitted useable Grade Level of Achievement data, 
reporting for 220,682 students, 3,380 of whom were not on a graded curriculum. The fields 
collected are as follows: 


All Students: 

e student name (surname and given name), 

e Alberta Student Number, and 

e enrolled grade, (defined as the grade to which the student was assigned) . 


GLA was collected for students on a graded curriculum as defined in the Alberta programs of 
study, in the following fields where applicable: 


e GLA in English Language Arts 

e GLA in French Language Arts - (French as the Language of instruction or Immersion 
students) 

e GLA in Mathematics 

e Grade English Language Arts Introduced 


Grade Level of Achievement in 2006-07 is defined as the grade level expressed as a whole 
number in relationship to the learning outcomes defined in the program of studies that teachers 
judged the student to have achieved at the end of the school year. A GLA Handbook (Alberta 
Education, 2006) was developed and distributed in the 2005-06 school year to facilitate pilot 
school participation in GLA reporting. 


The GLA Handbook encourages teachers to consider GLA assessment in relationship to the full 
range of formative and summative assessment information available to them over the course of 
the school year in making a professional judgment of the student’s grade level of achievement. 


Students not on a graded curriculum also had data submitted. “Not on a Graded Curriculum” 
was meant to indicate that the student’s program was restricted to learning outcomes that were 
significantly different from the provincial curriculum defined in the program of studies and were 
specifically selected to meet the student’s special needs as defined in the Standards for Special 
Education (Alberta Learning, 2004). The information collected was teachers’ ratings of 
students’ learning outcomes in three areas: communication skills, functional skills and academic 
readiness skills. “Communication skills” refer to the development of expressive and/or receptive 
communication. This could be verbal communication and/or alternative modes of 
communication. “Functional skills” refer to skills that would assist the student in developing 


independence in the home, school and community. “Academic readiness skills” refer to skills 
that would prepare the student for learning outcomes in the programs of study. 


Alberta Education staff used the Alberta Student Number to append data fields such as 
Provincial Achievement Test (PAT) results (both raw scores and achievement levels), student 
age, gender, number of school registrations, any additional special needs codes associated with 
the student, and starting date. Individual student identifiers were replaced with a discrete GLA 
data ID, leaving no personal identifiers in the dataset used in producing this report. 


All Charter School Authorities were required to report GLA in Language Arts and Math for all 
students in Grades | to 9. Jurisdictions were required to report GLA in Language Arts and Math 
for a minimum of one third of schools in Grades 1 to 9. Many jurisdictions reported more than 
one third of schools in the 2006-07 school year. Seventy-one authorities submitted valid GLA 
data in the 2006-07 school year. The table below outlines the number of schools submitting 
GLA data in each jurisdiction. 


Table 1 - Jurisdictions Reporting GLA 


Number of | Number of 


Jurisdictions Reporting 


Percent of 


Schools Schools Schools 
K-12 Reporting Reporting 
|Almadina School Society —— —“‘“C;é‘irTSSCCCCCUd TT CC‘TCTCNCC«CN000.0% | 
|AuroraSchoolLid, ——“‘“‘(SSCSTSC“‘(;$RSC*dDYTC NCCT 100.0% 
| Battle River Regional DivisionNo.31 | 84d 29.4% | 


Black Gold Regional Division No. 18 

Calgary Arts Academy Societ See |) aaa 100.0% 
Calgary Girls' School Socie = i = 100.0% 
Calgary Roman Catholic Separate School District No. 1 | 980 
Calgary School District No. 19 | 196 {196 | 100.0% 


Calgary Science School Socie 


Canadian Rockies Regional Division No. 12 ea | ieee 100.0% 


CAPE - Centre for Academic and Personal Excellence 
Institute 


Chinook's Edge School Division No. 73 29.7% 
13 


Clearview School Division No. 71 a. eee 55.0% 


Christ the Redeemer Catholic Separate Regional Division 
No. 3 

East Central Alberta Catholic Separate Schools Regional 2 25.0% 
Division No. 16 


East Central Francophone Education Region No. 3 a | eee a 25.0% 


3 
| Elk Island Catholic Separate Regional DivisionNo.41_ | 6 | S| 31.3% | 
2 


Foothills School Division No. 38 


Edmonton Catholic Separate School District No. 7 


Percent of 
Schools 


Number of 
Schools 


Number of 
Schools 


Jurisdictions Reporting 


Fort McMurray Public School District No. 2833 


Fort McMurray Roman Catholic Separate School District 
No. 32 


Fort Vermilion School Division No. 52 


Foundations for the Future Charter Academy Charter 
School Socie 


Golden Hills School Division No. 75 


Grande Prairie Roman Catholic Separate School District 
No. 28 


Grande Prairie School District No. 2357 
Grande Yellowhead Regional Division No. 35 
Grasslands Regional Division No. 6 


-_ 
Ne 
> — 
Oo i) 


—_ 
_ 


Greater North Central Francophone Education Region No. 
2 

Greater Southern Public Francophone Education Region 
No. 4 


Nl 
NIN 


Greater Southern Separate Catholic Francophone 
Education Region No. 4 


Greater St. Albert Catholic Regional Division No. 29 


we 
— 
S 
S 
S 
x 


35.3% 
High Prairie School Division No. 48 


Holy Spirit Roman Catholic Separate Regional Division 
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as 
| Holy Family Catholic Regional DivisionNo.37_ | 
A a I 
| Horizon School Division No.67_ | 818% 
150 7 7 
[Lethbridge School DistrictNo.51_ TOT 30.0% | 
| Medicine Hat School DistrictNo.76 | GT 87.5% | 
| New Horizons Charter School Society | 100.0% | 
Northern Lights School Division No. 69 
[Northland School DivisionNo.61 | TCT 273% | 
| Northwest Francophone Education RegionNo.1 [| 3 [T3338 % | 
| Peace River School DivisionNo.10_ TTC 838.3% 
| Red Deer School DistrictNo.104 | 27 6% I 
Rocky View School Division No. 41 


Jurisdictions Reporting Number of | Number of | Percent of 
Schools Schools Schools 


St. Paul Education Regional Division No. 1 


St. Thomas Aquinas Roman Catholic Separate Regional 7 
Division No. 38 


Sturgeon School Division No. 24 


> Provincially, 60% of all-schools submitted GLA data. This table has excluded Parkland School Division that was 
exempted for 2006-07, Moberly Hall Charter school which closed, and Boyle Street Charter school that had no 
grade 1-9 enrolment from the school counts resulting in a higher percentage of schools submitting GLA data. 


Grade Level of Achievement — Summary of Results 


Students on a Graded Curriculum 


There were 217,302 students on a graded curriculum included in this sample for the 2006-07 
school year. The students are roughly evenly distributed by enrolled grade with approximately 
11% of the students in each grade cohort. The table below shows the distribution of the GLA 
sample data by enrolled grade (Grades | to 9) compared to the Provincial’ distribution. 


Table 2 - Enrolled Grade Distribution 


Enrolled Grade Distribution 


Percent of Province Percent of 95% 
Enrolled Frequency Total Total Confidence 
Grade Interval® 

] 23,692 10.9% 42,176 10.6%  +/-0.47% 

2 23,929 11.0% 42,493 10.7%  +/-0.47% 

3 23,896 11.0% 42,656 10.7%  +/-0.47% 

4 23,954 11.0% 42,597 10.7%  +/-0.47% 

5 23,839 11.0% 43,858 11.0%  +/-0.47% 

6 24,582 11.3% 45,050 11.3%  +/- 0.46% 

7 24,240 11.2% 45,544 11.5%  +/-0.47% 

8 24,267 11.2% 45,790 11.5%  +/-0.46% 

9 24,903 11.5% 47,014 11.8%  +/-0.45% 
Total 217,302 100.0% 397,178 100.0%  +/- 0.15% 


Students could have been coded as severely disabled, mild/moderately disabled, gifted or ESL. 
Those students who were not coded as any of those categories are termed ‘non-coded’. The non- 
coded students make up the largest proportion of the sample as shown in the chart below. 


* In Tables 2 and 3, the “Province” columns refer to the data for the entire province. In the remainder of the report, 
prouinkaal comparisons refer to the provincial sample of GLA data. 

The estimated confidence interval is expressed as a percentage and indicates how confident the researcher wants to 
be (e.g. 95% confident) that the data collected accurately represents the results of the entire population. The 
objective is to obtain a sample large enough to ensure a 95% confidence that the results will be within +/- 3% 
around the mean. In this report sample sizes are very large and provide a high degree of confidence that sampling 
error is minimal. 


Table 3- Types of Student Codes 


Type of Student Codes 


GLA Percent of Provincial Percent of 95% 
Totals GLA Total Totals Provincial Confidence 
Total Interval 

Non-Coded 170,637 77.9% 313,612 79.0% +/- 0.17 
Severe Disabilities 5,303 2.4% 12,576 3.2% +/- 0.86 
Mild/Moderate Disabilities 16,769 7.7% 30,835 7.8% +/- 0.56 
Gifted 3,314 1.5% 5.221 1.3% +/- 1.31 
ESL —Canadian-born 11,795 5.4% 18,690 4.7% +/- 0.69 
ESL — Foreign-born 11,139 5.1% 16,244 4.1% +/- 0.71 
Total 218,957* 100.0% 397,178 100.0% +/- 0.13 


*This is higher than the total in Table 2 because some students have double codes. 


The distribution of students in each of the GLA results categories by subject is shown in the table 
below. 


Table 4 - All Students, Provincial 


All Students — Province 

| | Mathematics _——[_English Language Arts | __French Language Arts | 

Po ee Percent of total Percent of total Percent of total 
students enrolled (%) students enrolled (%) students enrolled (%) 


Table 5 - Students Non Coded, Provincial 


Students non coded (as mild/moderate, severe, gifted, or ESL) — Province 


| | Mathematics [English Language Arts | _French Language Arts _| 
students enrolled (%) students enrolled (%) students enrolled (%) 


° GLA NA refers to missing data, a “not applicable” situation, or “not available.” 


Severe Disabilities 

There were 5,303 students coded as severely disabled who had GLA reported for English 
Language Arts and Mathematics. There were 69 students coded as severely disabled who had 
GLA reported for French Language Arts. The chart below presents students having a severe 
disability code and their grade level of achievement. 


Table 6 - Severe Disabilities, Provincial 


Severe Disabilities — Provincial 


| Mathematics = English Language Arts French Language Arts 


Number of Percent of total Number 0 Percent of total Number of Percent of total 
students enrolled (%) students enrolled (%) students enrolled (%) 
g 


Students coded as having 


5) cd 
es 


More than half of students with a severe disability enrolled in Mathematics have a GLA equal to 
their enrolled grade (55.1%). In Language Arts 53.5% of students have a GLA equal to their 
enrolled grade. Mathematics and Language Arts are fairly similar in their GLA distribution. A 
very low proportion of French Immersion or French as the language of instruction students were 
coded as severely disabled. 


The chart below provides information on all students who were on a graded curriculum and were 
coded as severely disabled, by their disability type. The majority of students coded with a severe 
disability are those with an emotional/behavioural or physical/medical disability. Among the 
different types of severe disabilities, the distribution of students among GLA categories varies 
more widely than the overall severe disability distribution (see above table). Knowing the type 
of severe disability may aid in understanding the student distribution throughout GLA categories. 


Frequency | Percentage % at or % below % GLA % at or % below % GLA 
of Total above grade level NA above grade | grade level NA 
srade level level 


Severe Emotional /Behavioural Disabilit 58.9% | __35.9% 56.8% | 37.9% | 5.3% 


Severe Physical or Medical Disabilit 52.4% 7.7% 


Note: 1 ECS student was not shown on this table. The student is included in the remainder of the report as they are enrolled in a grade. 


Table 8 - Type of Severe Disability by Gender 


Type of Severe Disability by Gender 


Gender | Frequency | Percentage % at or 
of each above 
disability grade 
code level 


level 


: 
Prats [5 [arin | asanie | axer | nner | arer | sooner [ae 
[roa Cd Mate | aioe | rare | arm | aoa] eam | sam | same | oar | 
Pid remate [sn | mae | arr | asa | ram | osm | ame | 10% | 


Note: 1 ECS student was not shown on this table. The student is included in the remainder of the report as they are enrolled in a grade. 
* Asterisk indicates a statistically significant difference between genders for the GLA categories indicated. Differences are significant at the 0.05 level. 
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When the students coded as having a severe disability are shown by gender, the large difference 
between genders becomes apparent. A large proportion (77.4%) of these students were males, 
most of whom were students with an emotional/behavioural disability. 


Mild/Moderate Disabilities 

There were 16,769 students in the 2006-07 sample having mild or moderate disability codes in 
English Language Arts and Mathematics. In French Language Arts, 369 students were coded as 
having a mild or moderate disability. The chart below shows the mild or moderate students’ 
distribution across GLA categories. 


Table 9 - Mild/Moderate Disabilities 


Mild/Moderate Disabilities — Provincial 


| Mathematics | English Language Arts French Language Arts 
Number of Percent of total Number of Percent of total Number of Percent of total 
students enrolled (%) students enrolled (%) students enrolled (%) 


In both Mathematics and Language Arts, a slim majority of students have a GLA equal to their 
enrolled grade. 


Students coded as having 
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The chart below shows the breakdown of students on a graded curriculum having a mild or 
moderate disability by the type of disability. The largest proportion of students coded with a 
mild/moderate disability has a mild cognitive disability or a learning disability. Students having 
a mild or moderate cognitive disability compose the largest proportion of students attaining 
below their grade level. As in the severe disability chart, students who were ‘at or above grade 
level’ were combined into a single category due to the small number of students who were 
achieving ‘above grade level’. 


Table 10 - Type of Mild/Moderate Disability 


Type of Mild/Moderate Disability English Language Arts 


Frequency Percentage % at or % below | %GLANA % at or % below % GLA 
of Total above grade | grade level above grade grade level NA 
: level level 


Mild Cognitive Dissbilit 2,922 17.4% 21.6% 68.7% 21.0% 69.7% 
2 2 
2 1 
2 


Moderate Cognitive Disability | 93 | 0.6% | 8% | 64.5% 67.1% | ___ 22.6% 
Emotional/Behavioural Disabili 1,766 10.5% 27.9% 


8% 2% 


93 P ; a‘ 
; 69. 25 


Note: 2 students coded as ECS Developmentally Immature were removed from this analysis. They have been included in the rest of the report. 


The following chart shows each mild/moderate disability type by gender. 
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Table 11 - Type of Mild/Moderate Disability by Gender 


Type of Mild/Moderate Disability by Gender | Mathematics English Language Arts 


Gender Frequency | Percentage % at or % below % GLA % at or % below % GLA 
of each aboye grade level NA above grade level NA 
disabili srade level srade level 
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Note: 2 students coded as ECS Developmentally Immature were removed from this analysis. They have been included in the rest of the report. 
* Asterisk indicates a statistically significant difference between genders. Differences are significant at the 0.05 level. 
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Table 11 illustrates the gender differences in GLA categories across the mild/moderate disability 
types. The gender difference is most noticeable for students having a physical or medical 
disability. 


Gifted 

In this sample of students, 3,314 were coded as being gifted in English Language Arts or 
Mathematics. There were 156 students in French Language Arts coded as gifted. The chart 
below shows the grade level of achievement distributions for students coded as gifted. 


LZ 


Table 12 - Gifted Students, Provincial 


Students coded as gifted — Provincial 


| Mathematics | English Language Arts [French Language Arts _| 
Pe eee Percent of total Percent of total Percent of total 
students enrolled (%) students enrolled (%) students enrolled (%) 


GLA NA 


The general assumption with gifted students is that they tend to achieve and perform better than 
the population of students as a whole. According to the above frequency table, we can see that 
the majority of gifted students are performing equal to their enrolled grade level, and slightly 
more that 2% of students are ‘below grade level’ for each subject. Gifted students are 
performing better in Mathematics than in English Language Arts with approximately 7% more 
students having a GLA ‘above grade level’ in Math. 


English as a Second Language 

For English as a Second Language (ESL) students, there were two groups of student codes. The 
first are ESL students coded as Canadian-born (303). Of this group of students there were 
11,795 in English Language Arts and Mathematics and 353 students in French Language Arts. 
For ESL students who are coded as foreign-born (301), there were 11,139 in English Language 
Arts and Mathematics and 242 in French Language Arts. The distribution of both groups’ GLAs 
is shown in the charts below. 


Table 13 - Canadian-born ESL Students, Provincial 


Students coded as English as a Second Language — Canadians-born (303) - Provincial 


Ee aes 
students enrolled (%) students enrolled (%) students enrolled (%) 
FatAtevecnotedante | | ow wi omm| «fam 
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Table 14 - Foreign-born ESL Students, Provincial 


Mathematics 


Percent of total Percent of total 
enrolled (%) enrolled (%) 


The above tables show that both Canadian-born and Foreign-born ESL students follow a similar 
pattern to one another. Approximately 83-84% of each ESL group is attaining a GLA that is 
equal to or above enrolled grade in Math and about 72-78% are equal to or above grade level in 
ELA. 


Gender 

Students’ GLA was broken down by gender in order to observe any patterns that may emerge. In 
this sample of students there was a larger number of males than females. The charts below show 
students’ GLA by gender in each of the three subjects. 


Table 15 - Gender, Mathematics, Provincial 


Gender - Provincial Mathematics 


po Femate | Mae Cd 
ie ee ee 
students enrolled (%) students enrolled (%) 


Note: All of the above observed relationships were significant when measured by Chi square. 


Table 16 - Gender, English Language Arts, Provincial 


English Language Arts 


po emate Male i 
students enrolled (%) students enrolled (%) 


Note: All of the above observed relationships were significant when measured by Chi square. 


Gender - Provincial 
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Table 17 - Gender, French Language Arts, Provincial 


French Language Arts 


| Female | Male 
students enrolled (%) students enrolled (%) 
GLA below enrolled grade 6.7% 
GLA equal to enrolled grade 84.4% 
GLA above enrolled grade es ee 0.3% 


Note: All of the above observed relationships were significant when measured by Chi square. 


Gender - Provincial 


The three tables above show the frequency of female and male students in each of the GLA 
categories in Mathematics, English Language Arts, and French Language Arts. When testing for 
independence between genders, Chi-squared’ was used. There is a statistically significant 
difference between the genders in both Math and English Language Arts. 


Pope, Wentzel and Cammaert (2003) conducted a study which showed that larger gender 
differences were found in school-awarded marks than in the diploma exam marks in favour of 
females in almost all diploma courses. Math courses (both Math 30 Pure and Applied) were 
among those subjects where boys, while being outperformed by girls in school-awarded marks, 
did better than girls in diploma exam marks. Therefore, the question of gender differences 
between GLA and PAT results were of interest in the current report. 


The following tables illustrate the differences in GLA and PATs across the enrolled grades for 
each gender. It is quite interesting to note that, similar to the 2003 study, on GLAs, females 
outperform males to a statistically significant degree in both subjects in nearly all enrolled 
grades. The reverse is seen on the Math PAT results with males outperforming females to a 
statistically significant degree. However, in English Language Arts, females are outperforming 
males on the PATs. These findings point to a potential area for future research to investigate 
gender differentials and the causal factors associated with these differences. 


” Chi square is defined as “A nonparametric procedure for testing whether the observed frequencies of scores in 
different categories of a variable differ from the theoretically predicted frequencies” (Harris, 1995). 
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Table 18 - English Language Arts T-Tests 


English Language Arts GLA T-Tests English Language Arts PAT T-Tests 
Enrolled Gender 05-06 05-06 06-07 06-07 Enrolled Gender 05-06 05-06 06-07 06-07 Mean 
Grade N Mean N Mean Grade N Mean PAT WN PAT Score 
GLA GLA Score 
1 F 3,562 1.01 11,525 0.93* 
M 3,518 1.01 12,167 0.88* 
2 F 4,161 1.93* 11,704 1.90* 
M 4,356 1.89% 12,225 1.85* 
3 F 4,236 2.89* 11,641 2.88* 3 F 4,088 71.03* 10,790 69.72* 
M 4,511 2.84* 12,255 2.83* M 4,244 68.50* 11,537 66.72* 
4 F 4,281 3.86* 11,608 3.87* 
M 4,633 3.79* 12,346 3.79* 
5 F 4,535 4.81* 11,440 4.84* 
M 4,664 4.75* 12,399 4.76* 
6 F 4,280 5.80* 12,114 §.82* 6 F 4,039 68.36* 11,451 69.79* 
M 4,689 5.71* 12,468 5.74* M 4,304 64.90* 11,923 65.40* 
7 F 4,127 6.87* 11,790 6.88* 
M 4,350 6.75* 12,450 6.78* 
8 F 4444 7.88* 11,849 7.88* 
M 4,489 7.77* 12,418 Ide 
9 F 4,466 8.87* 12,259 8.87* 9 F 4,154 71.42* 11,337 68.36* 
M 4,571  8.77* 12,644 8.76* M 4,139 66.77* 11,742 63.25* 
* indicates a statistically significant difference between males and females. 
Table 19 - Math T-Tests 
Mathematics GLA T-Tests Mathematics PAT T-Tests 
Enrolled Gender 05-06 05-06 06-07 06-07 Enrolled Gender 05-06 05-06 06-07 06-07 
Grade N Mean N Mean Grade N Mean PAT N Mean PAT 
GLA GLA Score Score 
] F 4,024 1.01 11,525 0.95* 
M 4,080 1.01 12,167 0.93* 
2 F 4,293 1.95 11,704 1.93 
M 4,495 196 12,225 1.92 
3 F 4,250 2.92 11,641 2.90 3 F 4,083 33.12* 10,790 30.60* 
M 4,530 2.91 12,255 2.89 M 4,240 34.07* 11,537 31.54* 
4 F 4,274 3.88 11,608  3.89* 
M 4,640 3.87 12,346 3.86% 
5 F 4,521 4.84 11,440 4.85* 
M 4,660 4.83 12,399 4.82* 
6 F 4,266 5.83. ..12,114 . 5.82* 6 F 4,054 36.64* 11,451 34.47* 
M 4,670 5.80 12,468  5.80* M 4,335 37.60* 11,923 36.03* 
7 F 3,941 6.88* 11,790 6.87% 
M 4,212 6.83* 12,450 6.82* 
8 F 4,197 7.89* 11,849 7.86* 
M 4,254 = 7.80% 12,418  7.81* 
9 F 4,217 8.85* 12,259 8.82* 9 F 4,166 32.44 11,337 30.80* 
M 4,346 8.78* 12,644  8.76* M 4,162 32.35 11,742 31.36* 


* indicates a statistically significant difference between males and females. 
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Student Mobility 


When a student changes schools, he/she must learn to deal with a new physical, social and 
learning environment - new teachers, new classmates, possibly different sets of rules, different 
learning expectations - and may start at a different point in the curriculum than he/she left 
behind. It also takes some time for teachers to determine the student’s learning level, learning 
style, interaction skills, etc., and thus define the optimal program for his/her ongoing learning. 


These issues contribute to the findings, within the research literature, of a negative relationship 
between the number of times a student changes schools in a given period and his/her academic 
growth in that period. Other studies (Wasserman, 2001) of this relationship in an Alberta setting 
have supported these findings and suggest that additional research would be useful, to not only 
enrich the understanding of the relationship but to highlight any situations in which the negative 
impacts may have been mitigated by helpful strategies to support better transitions for students 
and schools. The GLA data provides such an opportunity for additional research. It permits an 
on-going analysis of the relationship between the number of school changes students have made 
and their current grade level of achievement, thus allowing for an assessment of the cumulative 
impact of mobility. 


Student mobility is captured by Alberta Education, once at the end of September and again in 
March, and compiled in the Student Information System (SIS). The Student Mobility Indicator 
(SMI) provides an indication of the number of times a student has changed schools since entry 
into the Alberta school system. The SMI is calculated by counting the number of different 
school registrations each student has up until the most recent calendar year. Students could be 
changing schools more frequently than is captured, thus the numbers shown are a conservative 
estimate of student mobility. All students start with an SMI of 1 as they have all been registered 
in at least one school. Student mobility is then broken down into two categories — high and low. 
In Grades 1-3, high mobility students are those having a mobility indicator of 2 or more. 
Students having a mobility indicator of 1 are considered low mobility. In Grades 4-6, high 
mobility students are those having a mobility indicator of 3 or more. Students having a mobility 
indicator of 2 or less are considered low mobility. In Grades 7-9, high mobility students are 
those having a mobility indicator of 4 or more. Low mobility students have a mobility indicator 
of 3 or less. In the following tables the two categories of mobility includes students on a graded 
curriculum in Grades | to 9. 


Table 20 - Student Mobility, Mathematics, Provincial 


Mobility - Provincial Mathematics 


g Low 


a Sos 
students enrolled (%) students enrolled (%) 
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Table 21 - Student Mobility, English Language Arts, Provincial 


Mobility - Provincial English Language Arts 


a es | es 2 
: students enrolled (%) students enrolled (%) 


Table 22 - Student Mobility, French Language Arts, Provincial 


Mobility - Provincial French Language Arts 


ee es 7 


Number of Percent of total Number of Percent of total 
students enrolled (%) students enrolled (%) 
GLA below enrolled grade 628 
GLA equal to enrolled grade 


The majority of students can be described as having low mobility (69.1% of all students with 
GLA data). One of the most salient observations from the above tables is that a greater 
proportion of high mobility students have a GLA below their enrolled grade level compared to 
low mobility students. This observation supports the hypothesis that mobility negatively affects 
student achievement. 


As noted above, student mobility was calculated based on the number of different school 
registrations a student had accumulated. Each student begins with an SMI of 1, indicating that 
they have been registered in one school. When a student registers in a different school, the SMI 
will increase by 1. The chart below lists the student mobility indicator values and the number of 
students having each SMI. 
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* 15 students are missing a student mobility indicator 


The chart above shows approximately 86% of students in our sample had 3 or fewer different 
school registrations. Bear in mind students in the table above are in Grades 1 through 9 and 
those students in the younger grades are the reason for the large number of students with a 
mobility indicator of 3 or less. The following chart displays SMI by GLA above or equal to 
enrolled grade as well as by GLA below enrolled grade for English Language Arts. 


Table 24 - Student Mobility Compared to GLA levels in English Language Arts 


Total 185,681 (85.5% 24,113 (11.1%) | 7,493 (3.4% 217,287* 


* 15 students are missing a student mobility indicator 


~The following two charts further examine the efzect of student mobility on grade level of 
achievement in English Language Arts. 
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Table 25 - Student Mobility and English Language Arts by Enrolled Grade 


English Language Arts — Student Mobili 


Enrolled | Mobility | GLA Equal or | GLA Below GLA NA Total 
Grade Above 
[High | 
9 


High 3,382 (43.7%) 685 (8.8% 296 (3.8%) | 4,363.(18.4%) 
15,661 (44.8% 1,633 (4.7% ,030 (5.8%) | 19,324 (81.6% 
6,104 (44.6% 1,120 (8.2% 358 (2.6%) | 7,582 (31.7% 
13,645 (45.5% 1,437 (4.8% 1,264 (4.2%) | 16,346 (68.3% 
7,411 (45.0% 1,420 (8.6% 227 (1.4%) | 9,058 (37.9% 
13,111 (46.9%) 1,402 (5.0% 323 (1.2%) | 14,836 (62.1% 
3,589 (44.1% 895 (11.0% 70 (0.9%) | 4,554 (19.0% 
OW 


| Low _[|_17,025 (46.7%) 2,059 (5.7% 316 (0.9%) | 19,400 (81.0% 
4,446 (44.0%) | _ 1,105 (10.9%) 100 (1.0%) | 5,651 (23.7% 
I 


igh 
w 
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The next graph shows the mean GLA values in English Language Arts for each grade broken 
down by the student mobility indicator. 


Mean GLA per enrolled grade as SMI increases in English Language Arts 


Grade 


Or NwWHUNDAN CO 
\ ee eee 


SMI 


* Categories with fewer than 20 students have been removed 
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In Grades | through 6 the majority of students have an SMI of 1 indicating that these students 
have not changed schools. In Grades 7 to 9 the majority of students have an SMI of 2 illustrating 
the usual pattern of moving to a new school for junior high. 


Table 26 - Student Mobility Compared to GLA levels in Mathematics 


Students’ GLA levels compared to Mobility Indicators in Mathematics 


Indicator Above 


pT 3,025 (74.1%) | 870 (21.3%) | 185 (4.5%) | 4,080 | 
| BT 775 (64.6%) | 335.(27.9%) | 907.5%) | 1,200 | 
pT 429(58.4%) | 229, 31.2%) | 77 00.5%) | 735 | 


* 15 students are missing a student mobility indicator 


Table 27 - Student Mobility and Math by Enrolled Grade 


Math- Student Mobili 


Enrolled | Mobility | GLA Equal or GLA Below GLA NA Total 
Grade Above 


rl 3,779 (86.6% 475 (10.9% 109 (2.5%) | 4,363 (18.4% 
17,922 (92.7% 989 (5.1% 413 (2.1%) | 19,324 (81.6% 
6,718 (88.6% 754 (9.9% 110 (1.5%) | 7,582 (31.7% 
15,168 (92.8%) 947 (5.8% 231 (1.4%) | 16,346 (68.3% 
7,739 (85.4% 1,128 (12.5% 191 (2.1%) | 9,058 (37.9% 
13,616 (91.8% 999 (6.7% 221 (1.5%) | 14,836 (62.1% 
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[Low | ___15,657 (90.2%) | __1,520(8.8%) | ___187 (1.1%) |_ 17,364 (70.6%) 
16,672 (90.5%) 
[High | ___—5,663 (73.8%) | 
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The mean GLA values for Math in each grade were plotted against SMI values to produce the 
following chart. Once again, aside from the first grade there is a gradual decrease in the mean 
GLA as the SMI increases. 


Mean GLA per enrolled grade as SMI increases in Mathematics 


Grade 
Sore NY WBN A ~I Cw 


~ SMI 


es 
* Categories with less than 20 students have been removed 


The following graph illustrates the effect mobility can have on achievement. 


; Percent of Students Below Grade Level by Mobility Category for each 
Grade 


25.0% 
20.0% 
15.0% 
10.0% 
5.0% 
0.0% 


Percent Below Grade 
Level 


Enrolled Grade 


—s— High Mobility a— Low Mobility | 


As the graph above illustrates, there is a noticeable effect between high and low mobility. There 
is a statistically significant difference between the mobility groups in Grades 4 through 9. This 
significance may indicate that mobility has a more pronounced effect as students age. It may also 
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be due to the likelihood of students having a higher mobility indicator as their enrolled grade 
increases. 


Correlations between SMI and GLA 

Kendall’s tau-b values were calculated in order to illustrate the relationship between SMI and 
GLA groupings. Kendall’s tau-b is an alternative non-parametric form of rank correlations. The 
tau-b is used as an inferential statistic to show the strength of those relationships. Tau-b is used 
to measure the association between student mobility and GLA. This particular test was chosen 
as it uses ordinal level data based on pair by pair comparisons. The chart below details the 
correlations. 


Table 28 - SMI by GLA, Tau-b Calculations 


SMI by GLA- Grade and 05-06 06-07 
Subject Tau-b Tau-b 
Eng. LA and SMI 0.096* 0.103* 
Math and SMI 0.126* 0.117* 


* correlation is significant at the 0.01 level. 


The above correlations between student mobility and GLA are significant at the 0.01 level in 
both years. They are relatively weak in strength, although this is to be expected as student 
mobility can affect a student’s GLA but is only one of many determining variables. The 
correlations are similar between the two years of data indicating the validity of the data. 


Students not on a Graded Curriculum (Modified Programmin 


There were 3,380 students not on a graded curriculum in this study. The students were assessed 
based on the degree of achievement in their Individual Program Plans (IPP’s) relative to their 
foundational skills, academic readiness skills and life skills. The results reported below should be 
considered preliminary as data will become more meaningful as multi-year trends become 
available. 


Table 29 - IPP Foundation Skills 
Foundation Skills 
eG ee 3 aoe 
Students Total Enrolled 
% 
All skills attained 
Most skills attained 1,078 
Some skills attained 
None of the skils tained SS 8 
| 8 


N/A 18.3% 
Total 


3,380 100.0% 
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Table 30 - IPP Academic Readiness Skills 


Academic Readiness Skills 


[ee 
Students Total Enrolled 
% 
Life Skills 


Table 31 - IPP Life Skills 
Number of Percent of 
Students Total Enrolled 
% 
397 
995 


Some skills attained 878 


None of the skills attained aaa 
N/A 


GLA and Enrolled Grade 
The following tables and graphs show GLA values by enrolled grade for all students, in English 
Language Arts and Math. N1 indicates students that are not yet at the Grade 1 level. 


Grade Level of Achievement by Enrolled Grade 


Grade One 


Grade 1 
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GLA Value 
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Grade Two 


Grade % of Total % of Total 
Level 


PNi | 333] 14% | 189] 0.8% | 
pdf 2,224] 9.3% | 1512] 6.3% | 


Ci ee 
1,623 
100.0% 


Grade Three 


% of Total % of Total 
| 0.4% | 


04% | 60] 0.3% | 
| 0.0% | OO] 0.0% | 
[00% | OO 0.0% | 


Grade Four 


Grade ELA % of Total % of Total 
Level 
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Grade Five 


| 2 
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1.7% 
Grade Six 
Grade % of Total % of Total 
Level 
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Grade Seven 
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Number of students 


Grade 5 
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GLA Value 


Grade 6 


23 4 5 6 7 8 9 
GLA Value 


Grade 7 


25,000 re 
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Grade Eight 


Grade 8 


Number of students 


N1 123 4 5 6 7 8 9 1011 
GLA Value 


Grade 9 


25,000 
20,000 
15,000 + 
10,000 
5,000 
0 
N1 12 3 45 6 7 8 9 1011 12 


GLA Value 


Number of students 


0.0% | _5 | 
0.0% | _0| 
| 1,136 | 


3.4% | 1,136 


0.0% 
5 


Using the student counts from the above tables, the number of students whose GLA is below 
grade level in each grade are shown in the graph below. From Grade 1 to 6 there are a higher 
number of students attaining below grade level in English Language Arts. In Grade 7 to 9 there 
are a higher number of students attaining below grade level in Mathematics. 
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Number of Students Below Grade Level by Enrolled Grade 


% of Total | Math | % of Total 


Number of Students Below Grade Level 


| sd | s2,320 1,465 6.9% by Enrolled Grade 


4 2,320 10.9% 8 # 3,000 
5 2,627 12.3% 3 8 2,000 
6 12.9% E 2 1,000 | 
11.4% = 0 ? 
9 


2,557 10.6% | 1,701 8.0% 
2,822 2,128 10.0% 4,000 


12.5% 123 45 678 9 
10.5% | 3,245 15.2% Gk Value 

100.0% 100.0% 

Correlations between GLA and Enrolled Grade by Sub-Groups of the Population 
Correlations between the students’ GLAs and enrolled grades were calculated using Spearman’s 
rho to determine the “goodness of fit” between GLA and enrolled grade related to students’ 
being “coded” or not. The correlation between the two variables reflects the degree to which 
they “move” together. In this case, a high positive correlation coefficient results when an 
increase in enrolled grade is mirrored by the same increase in grade level of achievement. As 
was expected, GLA was highly correlated to enrolled grade, meaning the enrolled grade of a 
student typically matches their GLA. In English Language Arts this relationship was strongest 
for the gifted students, while students with severe disabilities had the lowest correlation between 
GLA and enrolled grade. In Math, the relationship was strongest for the non-coded students 


while students with severe disabilities had the lowest correlation. A comparison to the 2005-06 
GLA correlations is shown in the tables below. 


Table 32 - Correlations between GLA and Enrolled Grade, ELA 


English GLA 


2005-06 Correlation 2006-07 Correlation 

Student Codes Coefficient Coefficient 
Non-Coded 991** 201s" 
Severe Disabilities 893%" .868** 
Mild/Moderate Disabilities 845** .896** 
Gifted .994** .994** 
ESL Canadian Born (303) .974** .966** 
ESL Foreign Born (301) 925°" 945** 


**Correlation is significant at the 0.01 level (2-tailed) 
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Table 33 - Correlations between GLA and Enrolled Grade, Math 


Mathematics 
2005-06 Correlation 2006-07 Correlation 
Student Codes Coefficient Coefficient 
Non-Coded .994* 992** 
Severe Disabilities .863** .874** 
Mild/Moderate Disabilities .854** .901** 
Gifted .980** .I84** 
ESL Canadian Born (303) 985** D75** 
ESL Foreign Born (301) .O74** O71* 


** Correlation is significant at the 0.01 level (2-tailed). 


The graphs (next pages) show GLA by enrolled grade for all students as well as sub-populations 
of students, in English Language Arts and Math. The mean GLA value is plotted against the 
enrolled grade to show graphically the degree to which students’ GLA reflects or departs from 
their enrolled grade relative to sub-group affiliations. | As anticipated, the first two graphs 
demonstrate the close relationship between enrolled grade and GLA for non-coded students. 


This pattern is parallel to the high percentage of students who attain the acceptable standard on 
provincial achievement tests. 


Non-Coded Students — English Language Arts 


Mean English LA GLA by Enrolled Grade - Non-coded Students 


—¢e— Enrolled Grade 
oi Mean GLA 


Mean GLA 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 
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Mean GLA 


Non-Coded Students — Mathematics 


Mean Math GLA by Enrolled Grade - Non Coded Students 


—e— Enrolled Grade 
~~~ Mean GLA 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 


Mean GLA 


Students with Severe Disabilities- English Language Arts 


Mean English LA GLA by Enrolled Grade - Severe Disabilities 


—¢— Enrolled Grade 
itt M ean GLA. 


1 2 3 4 5 6 d 8 9 


Enrolled Grade 
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Students with Severe Disabilities — Mathematics 


Mean GLA 


Mean Math GLA by Enrolled Grade - Severe Disabilities 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 


—e— Enrolled Grade 
—t-—- Mean GLA 


Students with Mild/Moderate Disabilities — English Language Arts 


Mean GLA 


Mean English LA GLA by Enrolled Grade - Mild/Moderate Dis abilities 


—e— Enrolled Grade 
itime Mean GLA 


1 2 3 4 5 6 i 8 9. 
Enrolled Grade 
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Students with Mild/Moderate Disabilities — Mathematics 


Mean GLA 


Mean Math GLA by Enrolled Grade - Mild/Moderate Disabilities 


1 2 3 4 5 6 
Enrolled Grade 


Mean GLA 


—e— Enrolled Grade 
~—~tit—~ Mean GLA 


Gifted Students — English Language Arts 


Mean English LA GLA by Enrolled Grade - Gifted 


1 2 3 4 5 6 
Enrolled Grade 


—e— Enrolled Grade 
~~ Mean GLA 
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Gifted Students — Mathematics 


Mean GLA 


Mean GLA 


Mean Math GLA by Enrolled Grade - Gifted 


—e— Enrolled Grade 
ott Mean GLA 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 


ESL Canadian-Born Students — English Language Arts 


Mean English LA GLA by Enrolled Grade - ESL Canadian Born (303) 


—e— Enrolled Grade 
~~ M ean GLA 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 
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ESL Canadian-Born Students — Mathematics 


Mean Math GLA by Enrolled Grade - ESL Canadian Born (303) 


Mean GLA 


Enrolled Grade 
L 
ESL Foreign-Born Students — English Language Arts 
Mean English LA GLA by Enrolled Grade - ESL Foreign Born (301) 
< 
16) —e— Enrolled Grade 
= 


1 2 3 4 5 6 at 8 


Enrolled Grade 
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ESL Foreign-Born Students — Mathematics 


Mean Math GLA by Enrolled Grade - ESL Foreign Born (301) 


—¢— Enrolled Grade 
omttti-—- M ean GLA 


Mean GLA 


1 2 3 4 5 6 7 8 9 
Enrolled Grade 


Discussion 
The graphs above yield some interesting observations: 


The relationship between enrolled grade and GLA for non-coded students is very close as 
was expected. 

The variance between enrolled grade and GLA tends to increase through the elementary 
grades and peaks at grade 6 and grade 9 for students with a severe disability. 

A similar pattern is seen for students with a mild/moderate disability, but the peak in grade 
nine is absent. 

Gifted students are more likely to have a GLA above enrolled grade in Mathematics than in 
Language Arts. 

Foreign-born and Canadian-born ESL students’ GLA distributions are relatively similar. 
Foreign-born ESL students seem to have a GLA on par with their enrolled grade in 


mathematics up until Grade 5. In English Language Arts the two variables diverge as early 
as Grade 2. 
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Birth Month-Combined Grades 


The following table shows the breakdown of student birth month. When grade levels are 
combined there is a relatively equal distribution of students across birth months, but with fewer 
students in the second January and February categories. 


Table 34 — Student Birth Month 


Percentage 

5.7% 

6.1% 
9.2% 
9.1% 
9.5% 
8.9% 
9.1% 
8.7% 


8.3% 
7.6% 
6.6% 
5.6% 
3.3% 
2.3% 
100.0% 


*Not all students were included in this table as some students were outside the parameters of analysis meaning they were much 
older than the students being examined in this table. 


The table above illustrates the distribution of students by birth month for all grades of students. 
The distribution was calculated by examining each grade individually and then combining all 
grades. Students in the 1" January and February are usually the oldest students in each grade, 
whereas students in the 2" January and F ebruary are the youngest in each grade. 


GLA and PAT by Age within Grade Cohorts 

Previous Alberta Education studies have indicated that there is a relative age effect between 
average PAT scores and birth month within grade cohorts, where older students tend to have 
higher average test scores than the younger students when measured by the z-score of average 
PAT results for each birth month group (Alberta Learning, 2001). 


A comparative analysis was undertaken using GLA data. The percentages of students ‘at or 
above’ their grade level in English Language Arts and Math were converted to z-scores® and 
plotted. (See graphs below). There is a noticeable age effect Grades 1 through 5, which is most 
pronounced in Grade 1. In Grade 1, 1“ January to October are statistically significant when 
compared to 2" January and 2™ February. 


* Z-scores or standard scores convert a distribution of a set of scores to a normal distribution that allows a more 
consistent analysis of the data. A z-score of +1.0 represents a score equal to one standard deviation above the mean. 
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The following graphs were produced using the z-scores for Grades 1 through 5, English 
Language Arts GLAs for students in the GLA dataset. 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 1 students 


Ist Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


Linear (z-score) | 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 2 students 


NOH Soom nv 
48S4S8S48S48SSS 


Ist Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


Linear (z-score) 


a7 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 3 students 


Ist 1st March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 
EE z-score Linear (z-score) 


The graph below demonstrates that when PAT scores are recoded into “percent at or above 
acceptable” to mimic the GLA data, the relative age effect remains. In Grade 3, March and April 
are statistically significant when compared to 2™ January. 


Z-score of the percent of students at acceptable or above on the Language Arts 
PAT, by birth month within cohort - Grade 3 students 


Ist 1st March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


~~ Z-Score Linear (z-score) | 
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Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 4 students 


Ist Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


|= Z-Score Linear (z-score) | 


Z-score of the percent of students at or above grade level in Language Arts, by 
birth month within cohort - Grade 5 students 


2.50 
2.00 
1.50 
1.00 
0.50 
0.00 
-0.50 
-1.00 
-1.50 


Ist Ist March April May June July Aug Sept Oct Nov Dec 2nd 2nd 
Jan Feb Jan Feb 


~~ Z-score Linear (z-score) 


In Grade 5, May is statistically significant when compared to August, September and October. 


Discussion 

The age effect, which illustrates that younger students achieve at a lower level than those 
students born earlier in the year, is apparent in English Language Arts in Grades 1 through 5. 
After Grade 5, the age effect tapers off and is no longer apparent. In this sample the age effect 
persisted longer than in previous analyses of GLA data where the effect dissipated by Grade 3. 
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GLA at a Glance 


The following table shows all groups of students and their GLA in Mathematics and English 
Language Arts including students for whom GLA data was either not available or not applicable. 


Table 35 - GLA Overall Summary Table, Including ‘GLA NA’ 


Overall Summary Table Mathematics English Language Arts 


Student Codes Frequency % At or 
Above 


6 


Note: The categories of ‘at grade level’ and ‘above grade level’ have been combined. There are not enough cases in the ‘above grade level’ 
category to be shown separately. 
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The following table shows all groups of students and their GLA in Mathematics and English 
Language Arts excluding students for whom GLA data was either not available or not applicable. 
In this table asterisks indicate significant differences between a specific cohort of students and 
the non-coded cohort. This analysis is important as it points to factors that may be limiting 
academic success for cohorts of students who are most at risk of under-achievement and thus 
would benefit from more focused systemic responsiveness to their learning needs. 


Table 36 - GLA Overall Summary Table, Excluding ‘GLA NA’ 


Above Above 
- Emotional/Behavioural 37.9% 2,771 59.9% 40.1% 
(ic cal 
- Physical or Medical 1,824 
| 50.0% 106] 39.6%] 60.4% 
| 20.9%] 66] 74.2%] 25.8%) 
Mild/Moderate Disability 58.5% 
- Learning Disabili 7,549 61.7% 56.1% 
2,638 23.9% 76.1% 23.2% 
- Communication Disabili 25.9% 2,235 62.3% 
Disabili 
| - Hearing Disability | 167 | 79.0% | 21.0% 68] 78.6%] 21.4%) 
- Moderate Cognitive 71 15.5% 84.5% 72 12.5% 87.5% 
- Visual Disabili 
Sas ee ees Gee A Pee eee 
hi ee Sie Peet eens PRN eee sameerie 
9.1%* 
| Student Mobility | 
iS) Ci a as ies Saea cael Ry ReeeetaT 
Note: The categories of ‘at grade level’ and ‘above grade level’ have been combined. There are not enough cases in the ‘above grade level’ 


category to be shown separately. 

*Denotes a significant difference at p<0.001 when compared to the non-coded sample of students. 

Note: Statistical testing was only done with the larger categories, disability types (severe and mild/moderate) were not tested against the 
non-coded sample. 
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Comparison of GLA and PAT data 


The GLA by PAT analysis demonstrates that GLA data can indeed supplement PAT data with 
reasonable reliability and validity. 


Correlations between PAT and GLA 

Kendall’s tau-b was used to measure the association between PAT and GLA. This particular test 
was chosen as it uses ordinal level data based on pair by pair comparisons. The chart below 
details the correlations for 2005-06 and 2006-07. 


The PAT and GLA variables were re-coded into the dichotomous categories: either ‘Below 
Acceptable’, or ‘At or Above Acceptable’ for PATs; and either ‘Below Grade Level’ or ‘At or 
Above Grade Level’ for GLA, and then the two dichotomous variables were compared. All 
relationships tested were at the p <.01 levels meaning they were significant. The p-value shows 
that the observed relationships are not due to chance. The following tables shows all tau-b 
values for the relationships tested and from this, one can conclude that the relationships are 
moderate in strength. A perfect relationship of 1.0 between GLA and PAT is neither an expected 
nor a desirable condition given the inherent differences in the evaluation designs which would 
underlie potentially different learning outcomes being measured with different assessment 
methods. 


Table 37 - PAT by GLA Tau-b Calculations 


2005-06 2006-07 

PAT by GLA- Grade and Subject Tau-b PAT by GLA- Grade and Subject Tau-b 

Gr. 3 Eng. LA 378 Gr. 3 Eng. LA 324 
Gr. 6 Eng. LA 406 Gr. 6 Eng. LA 337 
Gr. 9 Eng. LA 338 Gr. 9 Eng. LA 23 
Gr. 3 Math 388 Gr. 3 Math 342 
Gr. 6 Math 403 Gr. 6 Math 366 
Gr. 9 Math 399 Gr. 9 Math 409 


When comparing 2005-06 to 2006-07, the strength of the relationships has decreased. This 
decrease in strength may be attributed to the larger sample size in 2006-07 and therefore may be 
a truer picture than the correlations of the previous year. The only exception is the increase in 
strength of the relationship between the Grade 9 PAT and GLA results. 
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GLA by PAT Analysis-Comparisons Using Achievement Levels 


In order to further examine the relationship between the GLA data and provincial achievement 
tests (PATs) and provide an additional perspective on these relationships, both PAT and GLA 
data were again re-coded into the categories of ‘Below Grade Level and GLA NA’ and ‘At or 
Above Grade Level’ for GLA; and ‘Acceptable or Excellence’ and ‘Below Acceptable and 
Excused or Absent’ for PATs. These groupings were chosen based on the current Alberta 
Education standard for cohort reporting. The groups were then cross-tabulated with the 
hypothesis being that students who score at or above the acceptable level on PATs tend to be at 
or above grade level, and likewise those that score below acceptable tend to be below grade 
level. The following tables show some support for -he hypothesis, as 76-78% of the students in 
Language Arts and 65-80% of the students in Math who are at grade level are also at or above 
the acceptable level on the PATs. The data in the following tables includes all students from 
schools that submitted GLA data for 2006-07 in Grades 3, 6 or 9. 


Table 38 - Comparison of English Language Arts PAT and GLA 


Comparison of English Language Arts PAT and GLA 


Grade Level of Ach-evement — English Language Arts 
At or Above Grade —_ Below Grade Level Total 


Level or GLA NA 

PAT - Grade 3 Accept. or Excellence 77.6% 5.7% 83.4% 
English Language 17,304 1,276 18,580) 
Arts Below Accept., Excused 8.1% 8.5% 16.6% 
or Absent (1,801) (1,905) (3,706) 
Total 85.7% 14.3% 100.0% 
19,105 3,181 22,286) 
PAT - Grade 6 Accept. or Excellence 77.9% 5.4% 83.3% 
English Language (18,173) (1,266) (19,439) 
Arts Below Accept., Excused 8.1% 8.6% 16.7% 
or Absent (1,887) (2,014) (3,901) 
Total 85.9% 14.1% 100.0% 
(20,060) (3,280) (23,340) 
PAT — Grade 9 Accept. or Excellence 76.2% 4.3% 80.5% 

English Language 17,489 982 18,471 
Arts Below Accept., Excused 10.8% 8.7% 19.5% 
or Absent (2,483) (1,999) (4,482) 
Total 87.0% 13.0% 100.0% 

19,972 2,981 (22,953 


Note: All of the above observed relationships were significant when measured by Chi square. Bolded numbers 
represent inconsistent relationships between GLA and PAT data. 
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Table 39 - Comparison of Mathematics PAT and GLA 
Comparison of Mathematics PAT and GLA 


Grade Level of Achievement — Mathematics 
At or Above Grade —_ Below Grade Level Total 


Level or GLA NA 
PAT - Grade 3 Accept. or Excellence 79.6% 3.5% 83.2% 
Mathematics 17,753) (788 (18,541) 
Below Accept., Excused 9.5% 7.3% 16.8% 
or Absent (2,120) (1,632) (3,752) 
Total 89.1% 10.9% 100.0% 
19,873 2.420 22,293) 
PAT - Grade 6 Accept. or Excellence 74.0% 3.0% 77.0% 
Mathematics (17,279) (701) (17,980) 
Below Accept., Excused 13.4% 9.5% 23.0% 
or Absent (3,134) (2,226) (5,360) 
Total 87.5% 12.5% 100.0% 
(20,413) (2,927) (23,340) 
PAT — Grade 9 Accept. or Excellence 65.8% 3.0% | 68.7% 
Mathematics (15,056) (679) (15,735) 
Below Accept., Excused 18.0% 13.2% 31.2% 
or Absent (4,129) (3,022) (7,151) 
Total 83.8% 16.2% 100.0% 
(19,185 3,701 (22,886) 


Note: All of the above observed relationships were significant when measured by Chi square. Bolded numbers 
represent inconsistent relationships between GLA and PAT data. 


It should be noted, in reviewing the above two tables, that more students are categorized as 
‘below grade level’ in the PAT results than is true in GLA ratings. To illustrate, using the 
Mathematics table above, the following comparisons can be made: 


Table 40 - Students below acceptable, excused or absent on the Mathematics PAT 
Total # of # Students At or Above 


Percent of Students At or Above 
Students GLA Grade level GLA Grade Level 


PAT Math 3 3,152 2,120 56.5% 
PAT Math 6 5,360 3,134 
PAT Math 9 4,129 57.7% 
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Table 41 - Students below grade level or GLA NA on GLA Mathematics 


Students Excellent on PAT or Excellent on PAT 


The following tables illustrate students below grade level on GLA and below acceptable on PAT 
in English Language Arts. 


Table 42 - Students below acceptable, excused or absent on the English Language Arts PAT 


Test Total # of # Students At or Above | Percent of Students At or Above 


Table 43 - Students below grade level or GLA NA on English Language Arts GLA 


Test Total # of # Students Acceptable or | Percent of Students Acceptable 
Students Excellent on PAT or Excellent on PAT 
PAT ELA 3 1,276 40.1% 


il 
PAT ELA 6 1,266 38.6% 
PAT ELA 9 2,981 32.9% 


It would appear from the above tables that while nearly half of the students who do not attain the 
acceptable standard on the PAT are considered at or above grade level, a smaller percentage of 
those who are considered below grade level are able to attain the acceptable standard on the 
PAT. This suggests that in terms of evaluating acceptable progress, the PAT is a more difficult 
standard to attain than is the GLA. The two assessments can be expected to demonstrate some 
variance because they are different forms of assessment, but both are designed to assess whether 
a student has met grade level standards. Therefore, one might expect that the variability would 
be seen as much in one direction as the other. The fact that this is not true suggests several 
possibilities: 1) that it may be more difficult for teachers to assign a ‘below grade level’ 
evaluation to one of their students than is the case for markers of the PAT assessments, 2) 
student performance on PATs may be attenuated by test anxiety; 3) students may perform better 
on many assessments over time than on a single paper and pencil test, or 4) a combination of 
these factors. 
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Analysis of Students Below Grade Level 


In this section the ratings given by teachers through the GLA are further compared to PAT 
results in Grades 3, 6 and 9. In each case, it is possible to identify the students who are rated as 
below grade level by their teachers (GLA) and those rated as below the acceptable standard by 
the PAT. 


According to classical test theory (Crocker and Algina, 1986) all assessments are subject to some 
degree of test error. Given this fact the more achievement data available the clearer the true 
picture of a student’s achievement becomes. One would expect some differences as well as 
complementary relationships in the designation of individuals in the two ratings provided by 
PAT and GLA data. Classroom assessment is based on an array of assessment methods over 
time for the GLA rating, ideally measuring the full and complete range of learning outcomes; 
whereas the PAT is a single, highly valid and reliable, selected and constructed response test that 
typically measures between 66% and 95% of the curricular outcomes in the tested subjects’. The 
PAT’s are very likely the single best criterion-referenced assessment instrument available to the 
classroom teacher. Since the objective of both methods is to measure and provide evidence on 
how well a student is achieving as compared to the learning outcomes in the program of studies, 
one would expect a generally positive relationship between the students identified as “below” by 
both methods, and that is what we see in this data set. Where anomalous or inconsistent 
relationships are observed it presents the occasion to ask why and to delve deeper into the data to 
help understand why the measures differ. 


An examination of the GLA pilot data for English Language Arts and Mathematics shows that 
this assumption departs most dramatically for Math 9. 


Students Below Grade Level and Below Acceptable on PAT in 
English Language Arts 


ELA 3 ELA 6 ELA 9 


——-PAL “"tLA 


° These figures were obtained from a discussion paper presented to the Program and Accountability Advisory 
Committee, November 8, 2007. 
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Note: The data presented in the above tables are discrete data points. The lines were included to emphasize the patterns of the 
data. 


Students Below Grade Level or Below Acceptable on PAT in 
Mathematics 


35% 
30% 
25% 
20% 
15% 
10% 

5% 

0% 


Math 3 Math 6 Math 9 


—eo— PAT ~—~#— GLA 


Note: The data presented in the above tables are discrete data points. The lines were included to emphasize the patterns of the 
data. 


Discussion 

A primary reason for provincial aggregation of Grade Level of Achievement data is evaluation of 
education programs such as special education, English as a Second Language, etc. This 
observation is particularly relevant for those grades that do not have PAT testing where GLA can 
provide data that would otherwise not be available. Additionally, it is useful to be able to 
supplement PAT data with GLA data in Grades 3, 6 and 9 as the added advantage would be 
broader and richer data to inform program evaluation related decisions. An example is the 
analysis of students below grade level or the analysis of students missing PAT data found on the 
next few pages. 


Further, the fact that the tau-b values show moderatz strength lends credibility to the process of 
collecting GLA. A perfect relationship of 1.0 between GLA and PAT is neither an expected nor 
a desirable condition given the inherent differences underlying the evaluation designs. PAT data 
are derived from a single paper and pencil test whereas GLA data are based on numerous and 
more dynamic observations over time, and thus should be a much richer method of assessment, 
which one could reasonably assume to produce, positively correlated albeit different data than a 
PAT result. Considered together, PAT and GLA data provide a richer and more comprehensive 
picture of student achievement. 
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GLA for Students Missing PAT Data 


In order to determine whether GLA provides useful data for students for whom PAT data was 
unavailable, GLA data for these students was analyzed. Relative to the Grade 3, 6 and 9 students 
included in the 2006-07 GLA data set, the following table defines the breakdown of student 
categories and percent of students lacking PAT results and the corresponding GLA results. 


Table 44 - Student Missing PAT Data Grade 3 


Student Code Groups 


Non Severe Mild/Moderate | Gifted ESL - ESL —- All 
Coded Cdn-born Foreign- Students 
born 


Total Eligible 17, il es tel a — _ 642 1,320 22,376 
Grade 3 
Eng lish to write PAT 


°% of students 96.1% Loe 1% [ail 5% ae ™% oe 9% 81.7% 93. 0% 
writing PAT 
Students 


%not writing 3.9% 35.9% 27.5% 1.3% 71% 18.3% 7.0% 
rade be 
students 


Below Grade 
with no (aval 
PAT data At Grade Level 
or Above 


No GLA a ae eee 
% with no 19. Rad 14. | eee 6. a6] me 4% 18. i 
data having 
info using 

PAT data were previously available. Further, when GLA data are substituted, data are 

unavailable for only 5.1% of the sub-population coded severe (compared to 35.9% missing PAT 


GLA 
% of students = 4% Bist) 7% ae 5% 100.0% 76.1% ial ool a! 
GLA 
% of Grade 3 students having 99.2% | 94.9% 98.2% 100.0% 98.3% 92.8% 98.7% 
info using GLA, PAT or both 
data). Similarly, using GLA means data are unavailable for only 1.8% of the students in the 
dataset coded mild/moderate (compared to 27.5% using PAT data). Similar findings for Grades 


with no PAT 
Grade 3 English Language Arts GLA data provided data for 82% of the population for whom no 
6 and 9 in English Language Arts are summarized below. 


Language 
Arts 
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Table 45 - Students Missing PAT Data Grade 6 
Student Code Groups 


Non Severe Mild/Moderate | Gifted ESL — ESL —- All 
Coded Cdn-born Foreign- Students 
born 
arate Fs Rk ss el 
Eng lish to write PAT 


% of students 95.8% 54.5% 73.9% 99.2% 91.6% 86.1% 92.2% 
writing PAT 


At Grade Level 
or Above 


% with no 17.1% ia! i ee 14.3% 28.7% 12: Ps 
GLA 


% of students : ma 3% a 3% 100.0% 85.7% 71.3% ss 5% 
with no PAT 
data having 
info using 
GLA 
using GLA, PAT or both 


Table 46 - Students Missing PAT Data Grade 9 


Language Arts, 
Students 
Grade 6 
students with 
no PAT data 


Student Code Groups 


Severe Mild/Moderate | Gifted ESL — ESL - All 
Cdn-born Foreign- Students 
ep 
to write PAT 


English % of students 94.5% 46.5% eal 6% Hal 6% Bia) 3% 83.1% 90.9% 
Language Arts | writing PAT 
Students 
Grade 9 %not writing 5.5% 53.5% 30.4% 16.9% 
: PAT 
students with 
Below Grade 
no PAT data Level 
At Grade Level 
or Above 
eawe | } “| | 497 
2 23.8% 
data having 


| S| 55 
% 81.5% 
info using 


GLA 
76.7% 100.0% 90.2% 69.9% 76.2% 
GLA 


% of students 
% of Grade 9 students having info 98.6% 90.1% 93.0% 100.0% 99.1% 94.9% 97.8% 
using GLA, PAT or both 


with no PAT 
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As the charts above demonstrate, GLA provides data for the majority of students who would 
otherwise not have any data provided. 


Analysis of Grade ELA Introduced for FLA Students 


In the 2006-07 GLA pilot, schools were asked to enter the grade English Language Arts was 
introduced to French students. The following table shows the distribution of the grade in which 
ELA was introduced. 


Table 47 - Grade ELA Introduced Distribution 


Grade ELA Intro Frequency Percent 
1,471 17.0% 
2 2,495 28.8% 
3 4,535 52.3% 
4 34 0.4% 
5 11 0.1% 
6 14 0.2% 
7 86 1.0% 
8 19 0.2% 
9 8 0.1% 
Total 8,673* 100.0% 


*There were 4,013 students who did not have Grade ELA Introduced information 


The majority of students had English Language Arts introduced in Grades 1 through 3. The 
following tables illustrate the French and English Language Arts GLA compared to the grade in 
which ELA was introduced. The tables only contain those students who had ELA introduced in 
Grades | through 3 as they are the largest group of students. 


Table 48 - Grade ELA Introduced compared to French Language Arts GLA 


Grade ELA GLA Equal or GLA Below GLA NA 
Intro Above 


| 1,277 (86.8% 194 (13.2% 0 (0.0% 1,471 
2,445 (98.0%) 50 (2.0%) 0 (0.0%) 
4,299 (94.8%) 235 (5.2%) 1 (0.0%) 4,535 


Note: All of the above observed relationships were significant when measured by Chi square. 


Count 


Table 49 - Grade ELA Introduced compared to English Language Arts GLA 


Grade ELA GLA Equal or GLA Below GLA NA Count 
Intro Above 


ae ees 1,271 (86.4% 200 (13.6% 0 (0.0% 1,471 
2,434 (97.6% 61 (2.4%) 0 (0.0%) 
3 4,358 (96.1%) 176 (3.9%) 1 (0.0%) 


Note: All of the above observed relationships were significant when measured by Chi square. 
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As the above tables illustrate, there are statistically significant relationships between the grade 
ELA introduced categories. When examining the G_A results for both FLA and ELA, students 
with an ELA introduced of Grade 2 seem to be the largest proportion of students attaining a GLA 
equal or above their enrolled grade. 


Due to the variance of the grade in which ELA is introduced as shown in the above frequency 
table, it was decided that this field would continue to be reported in future GLA data submissions 
for French students. 


Analysis of Students with Multiple Years of GLA Data 


There were 64,170 students involved in both the 2005-06 and 2006-07 GLA data collections. 
There were 82,390 student records submitted in 2005-06 and 220,682 in 2006-07 with 78% of 
the 2005-06 student cohort having data for both years. The following analysis is based only on 
those students for whom GLA data is available for both years and is presented only to illustrate 
the type of trend analysis that will become viable as the GLA initiative becomes implemented 
over several years. 


Students enrolled grade in 2005-06 and 2006-07 are shown below. 


Enrolled Grade 


2005-06 2006-07 
Enrolled Frequency Percent Frequency Percent 
Grade 
] 7,929 12.4% 104 2% 
y 8,234 12.8% 7,892 12.3% 
3 8,210 12.8% 8,200 12.8% 
4 8,225 12.8% 8,212 12.8% 
2) 8,424 13.1% 8,227 12.8% 
6 7,535 11.7% 8,404 13.1% 
7 7,684 12.0% 7,558 11.8% 
8 7,898 12.3% 7,665 11.9% 
9 31 0% 7,908 12.3% 
Total 64,170 100.0% 64,170 100.0% 


The following table shows the English Language Arts GLA codes for all students submitting in 
both years. 
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Table 50 — All Students, English Language Arts 
All Students - English Language Arts 


2005-06 2006-07 


Percent of Percent of 
Murer el ial enrotted | Ome teal enrolled 
students % students %, 


GLA below enrolled grade 


*Indicates a significant difference between 2005-06 and 2006-07 


Percent of Students in Each of the ELA GLA Categories 


GLA below GLAequalto GLA above GLA NA 
enrolled grade _ enrolled grade enrolled grade 


There is a drop in the number of students attaining the above grade level category and there is an 
increase in the number of students attaining below grade level. In Mathematics the same patterns 
emerge. 


Table 51 — All Students, Mathematics 


All Students - Mathematics 


2005-06 2006-07 


Percent of Percent of 
Number of total enrolled Number of total enrolled | 
students % students of 


Total 100.0% 


*Indicates a significant difference between 2005-06 and 2006-07 
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Percent of Students in Each of the Math GLA Categories 


GLA below GLAequalto GLA above GLA NA 
enrolled grade enrolled grade enrolled grade 


Comparison of English Language Arts, 2005-06 to 2006-07 


To better understand the student’s changes in grade level of achievement from 2005-06 to 2006- 


07, the following tables were produced. 


Table 52 — 2006-07 Results for Students At or Above Grade Level in 2005-06, 
English Language Arts 


Students At or Above Grade Level in 2005-06 - English Language Arts 


eee eee" eee 
students enrolled (% 


GLA NA 
[Tot SSI | 100.0% 


Table 53 — 2006-07 Results for Students Below Grade Level in 2005-06, English 
Language Arts 


Students Below Grade Level in 2005-06 - English Language Arts 
pO 2006-07 
Percent of total 


GLA NA 
7,348 100.0% 
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Comparison of Mathematics, 2005-06 to 2006-07 


Table 54 — 2006-07 Results for Students At or Above Grade Level in 2005-06, 
Mathematics 


eT 206-07 
students enrolled (% 


Table 55 — 2006-07 Results for Students Below Grade Level in 2005-06, 
Mathematics 


Students Below Grade Level in 2005-06 - Mathematics 
2006-07 


eae, 
students enrolled (% 
5 
8 


GLA NA 
100.0% 


From the above tables it is apparent that in Mathematics, most students who were at or above 
grade level in 2005-06 attained the same categorization in the following year (92.8%). For 
students categorized as below grade level in 2005-06, 31.8% achieved a GLA equal to or above 
their enrolled grade in the following year. A similar pattern is shown for English Language Arts. 


Change in GLA by Enrolled Grade 

The following tables show students’ GLA in English Language Arts and Mathematics for 2005- 
06 and 2006-07. In order to be part of this analysis, students had to have advanced an enrolled 
grade from 05-06 to 06-07. Five hundred (500) students were excluded from this analysis. 


Grade One to Grade Two 
| ss Grade 1- 05-06 Grade 2 — 06-07 


Grade ELA % of Total % of Total 
Level 


Grade 1 - 05-06 Grade 2 — 06-07 


5 
l | 6345| 81.1] 847 | 

Ea ae) ee ee ee 2 

Cs ae ; 

| Missing | 


Grade Two to Grade Three 


0.3 
1.7 


84.7 


ato 


108 iG ir 
3.0[ 110 


Grade 4 — 05-06 Grade 5 — 06-07 


Grade | ELA % of Total % of Total 


| Grade 2 - 05-06 | 
Grade | Math % of sisal ead % of Total 
. aS 
1.1 f 
389 4.8 F 


a g 


Grade 3 — 05-06 Grade 4 — 06-07 


| 

Grade % of Total % of Total 
Level 

[NI 


a | 42] OS | 
454 5.6 
| 88.2 | 


Ca Ee ee ee eee 
| Missing | _136{ SS 1.6{ 53] 0G | 
Total | 8163] 100.0] 8.163 | 100.0 | 


Grade 4 — 05-06 Grade 5 — 06-07 


aaa % of Total % of Total 
Level 
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Grade Five to Grade Six 
Grade 5 — 05-06 Grade 6 — 06-07 


% of Total 


8,3 


b) 


haat 5 
Missing |_124| 1.7] 160 | 
| 7,496 | 


7,496 


ELA 


o 


Missing 116 
Total 7,629 


| 
=e 
| 116 | 
| 7,629 | 


[| Grade 5 — 05-06 Grade 6 — 06-07 


Grade | Math % of Total | Math 


[4 | 474 | 


jo | 7 03 |7244 | 86.6 
as a es ee ee 
a a ee ee 9 
Lr a ) a) 


fic > | Grade 6 — 05-06 Grade 7 — 06-07 


Grade % of Total % of Total 
Level 


Fs a) a) a 
Ee a ee ee ee 


Ca a a) ae EY 
|Missing| 159] 2.2] 298] 4.0 
| Total | 7,496] 100.0] 7,496 | 100.0 | 


Grade 7 — 05-06 Grade 8 — 06-07 


Lessa el sed noah eval 
aS |) a 2 ee) ee 2 
a Ta a ee 
| 6] 
2 ee ae ee 

3 | 6,228| 81.6 | 


n _ 
fon) 
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aa Grade 8 — 05-06 Grade 9 — 06-07 


al ting 
Level 

a rs a a ee 2 ee 
ce ee 5 es) es ee) 
a es ee |) ee ee 
es ed 
88 30 
Ce a 
as a a 
[8 | 6686 85.0] 758] 9.6 | 
oT tO 6395] 81.3 | 
rae | 0 0.0 | | 
504 6.5 


The tables above illustrate interesting patterns of grade level achievement. For students who are 
evaluated as being below grade level, there are many that have a grade level of achievement that 
is more than one grade below their enrolled grade. ‘t is interesting to note that in Mathematics, 
every group of students in the 2005-06 cohort, the percent of students at grade level decreases 
from 2005-06 to 2006-07. 


Conclusions 

This report described the processes and outcomes associated with the 2006-07 Grade Level of 

Achievement (GLA) pilot data collection, data management and data analysis. The key 

questions addressed by this report were 1) can GLA data be collected and reported on a 

consistent basis, and 2) does the data contribute to our knowledge base regarding student 

achievement? Results of this and previous studies of GLA pilots demonstrate that the answer to 
these two questions is yes and is demonstrated by the following key findings: 

e The degree of jurisdiction participation in the 2006-07 pilot was much higher than expected. 
A participation rate of 33 % was expected; however, the data submitted represents 60% of 
schools. 

e The error rate for data submission was low wich only 2.2% of the files submitted having 
errors. 

e Compared to the 2005-06 data, the 2006-07 data demonstrates similar results for the total 
cohort and for specific sub-groupings of GLA data. Patterns of data distribution observed in 
2005-06 with 82,000 students were also observed in the 2006-07 data. 

e The data analysis demonstrates many interesting relationships. For example: 

o The large difference between Math 9 Provincial Achievement Test data and GLA 
continues to be evident. 

o Significant variations are observed in GLA between student sub-groups. The 
GLA at a Glance (Table 36) illustrates this. 

o A much higher percentage of students coded as gifted are assessed above grade 
level in Math than in English Language Arts or French Language Arts. 
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When comparing foreign-born and Canadian-born students, both groups of 
students perform similarly in Mathematics but in English Language Arts, 
Canadian-born students are at an advantage. 

Although a serendipitous finding, there is a disproportionate ratio of male: female 
severely coded students. There are nearly four times as many males as females 
coded as severely disabled. 

When comparing GLA and PAT data, females outperform males in GLA to a 
statistically significant degree in both subjects in nearly all enrolled grades. The 
reverse is seen on the Math PAT results with males outperforming females to a 
statistically significant degree. In English Language Arts, females are 
outperforming males on the PATs. 

A greater proportion of high mobility students (i.e. those who have changed 
schools more frequently than other students) have a GLA below their enrolled 
grade level. 


e When examining mean GLA compared to enrolled grade, interesting patterns emerge: 


fe) 


The relationship between enrolled grade and GLA for non-coded students is very 
close as expected and is generally aligned with PAT results. 

The variance between enrolled grade and GLA tends to increase through the 
elementary grades and peaks at grade 6 and grade 9 for students with a severe 
disability. 

A similar pattern is seen for students with a mild/moderate disability, but the peak 
in grade nine is absent. 

Gifted students are more likely to have a GLA above enrolled grade in 
Mathematics than in Language Arts. 

Foreign-born and Canadian-born ESL students’ GLA distributions are much 
closer together this year than was seen in the 2005-06 data. 

Foreign-born ESL students seem to have a GLA on par with their enrolled grade 
in Mathematics up until Grade 5. In English Language Arts the two measures 
diverge as early as Grade 2. 


¢ The age effect'® is apparent in English Language Arts in Grades | through 5. After Grade 5, 
the age effect tapers off and is no longer apparent. This relationship is more pronounced in 
the 2006-07 GLA data than in 2005-06, possibly due to the larger number of students in the 
2006-07 data. 

e There are moderate strength correlations between PATs and GLA (Grades 3, 6 and 9). This 
demonstrates a reasonable degree of concurrent validity of the GLA data. 

e Using GLA and PAT information, GLA can provide important information for students that 
would not otherwise be available for students in grades not tested by PATs. 


Future Analysis 


As noted in the Limitations section of this report, given the pilot nature of the data presented, the 
findings must be considered preliminary and can provide only tentative conclusions regarding 
levels of student achievement and factors that affect achievement for specific student cohorts. 


'° Age effect is defined as older students in a grade tending to have higher average test scores than the younger students in that 
same grade when measured by the z-score of average PAT results for each birth month group (Alberta Learning, 2001). 
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Nevertheless, this report points to a number of potential research questions that may require 

further analysis as GLA reporting becomes more confirmatory as full implementation is attained 

in the 2007-08 school year. Among the questions that may require investigation are: 

e Does the grade 9 PAT in mathematics or the GLA for mathematics 9 represent the best 
predictive validity for subsequent high school math achievement? 

e What are the reasons for classroom-based assessment providing higher marks for female vs. 
male students in mathematics contradictory to PAT marks? 

e Why are gifted students more likely to have a GLA above their enrolled grade in 
mathematics than in Language Arts? 

e Why is the proportion of male students coded with a severe disability so much higher than 
female students? 

e What strategies might be most effective in offsetting the negative effect of mobility on 
student achievement? 

e What pedagogical adaptations might be helpful in mitigating the age effect in the early 
grades? 


In addition, in future years the 2006 census data will be available for analysis. At that time, the 
census data will be compared to the 2007-08 GLA data to provide an indication of the extent to 
which variables external to the school influences student achievement. To create the most 
accurate comparison of the census data and the GLA data, the census data will be broken down 
by enrolled students’ postal code. 


The organization of census data from 2001 is largely the same as that from 2006 with the main 
difference being Mother’s level of education. In 2001, the variable was Mother’s years of 
education. In 2006, the variable is now Mother’s level of education. 


Once data is available, the following analysis will be done: 
e Jurisdiction by Mother’s level of education 

e Jurisdiction by Avg. family income/LICO 

e Jurisdiction by rent/own 


The SES variables will then be compared to GLA to see if any patterns emerge. The following 
tables will show these results: 

e GLA by Mother’s level of education 

e GLA by avg. family income/LICO 

e GLA by rent/own 
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